A Simple Proof for the Iteration Complexity of the Proximal Gradient Algorithm

We study the problem of minimizing the sum of a smooth strongly convex function and a non-smooth convex function. We consider solving this problem using the proximal gradient (PG) method, which at each iteration uses the proximal operator with respect to the non-smooth convex function at the intermediate iterate obtained using the gradient with respect to the smooth strongly convex function. We introduce a simple novel analysis and show that the PG algorithm attains a globally linear convergence rate provided that the step size is sufficiently small. Consequently, we obtain iteration complexity results for the PG method. We also extend our analysis to study an inexact proximal method, called the proximal incremental aggregated gradient method, and show that this method is globally convergent with a linear rate.